Image processing apparatus and image processing method

ABSTRACT

An image processing apparatus includes a processing unit configured to perform, on a luminance distribution acquired in at least one direction crossing a vessel region in a fundus image of a subject&#39;s eye, a first smoothing operation for each first size and a second smoothing operation for each second size smaller than the first size; a first identifying unit configured to identify a position of a vascular wall in the fundus image on the basis of the luminance distribution obtained by performing the first smoothing operation; and a second identifying unit configured to identify positions of inner and outer boundaries of the vascular wall in the fundus image on the basis of the luminance distribution obtained by performing the second smoothing operation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation, and claims the benefit, of U.S.patent application Ser. No. 15/202,233 filed Jul. 5, 2016 (now U.S. Pat.No. 9,980,637), which claims priority from Japanese Patent ApplicationNo. 2015-136387 filed Jul. 7, 2015. Each of U.S. patent application Ser.No. 15/202,233 and Japanese Patent Application No. 2015-136387 is herebyincorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing apparatus and animage processing method used in ophthalmic practices.

Description of the Related Art

Subjects' eyes are often examined for the purpose of early diagnosis andtreatment of lifestyle-related diseases and diseases that are leadingcauses of blindness. A scanning laser ophthalmoscope (SLO), which is anophthalmic apparatus using the principles of confocal laser microscopes,is configured to raster-scan a laser beam (measuring beam) across afundus and quickly acquire a high-resolution planar image from theintensity of light returned from the fundus.

By detecting only light that has passed through an opening (pinhole),only returned light at a particular depth can be converted into animage, which has a higher contrast than images that can be acquired by afundus camera.

Hereinafter, an apparatus configured to capture such a planar image willbe referred to as an SLO apparatus, and the planar image will bereferred to as an SLO image.

By increasing the diameter of a measuring beam in the SLO apparatus, ithas become possible in recent years to acquire an SLO image of a retinawith higher lateral resolution. However, as the diameter of themeasuring beam increases, degradation in signal-to-noise (S/N) ratio andresolution of the SLO image caused by aberration of the subject's eyehas become a problem in acquiring an SLO image of the retina.

As a solution to this, an adaptive optics SLO apparatus including anadaptive optics system has been developed. The adaptive optics system isconfigured to measure the aberration of the subject's eye with awavefront sensor in real time, and correct, with a wavefront correctiondevice, the aberration of the measuring beam or returned light occurringin the subject's eye. With this adaptive optics SLO apparatus, an SLOimage with high lateral resolution can be acquired.

This SLO image with high lateral resolution can be acquired as a movingimage. For example, to noninvasively observe the circulation of blood,retinal vessels are extracted from each frame, and the transfer rate ofblood cells in capillaries is measured. Also, to evaluate a relationwith a visual function using the SLO image, visual cells P are detected,and the density distribution and arrangement of the visual cells P aremeasured. FIG. 6B illustrates an SLO image with high lateral resolution.This SLO image allows observation of a low-luminance region Qcorresponding to the position of the visual cells P and capillaries, anda high-luminance region W corresponding to the position of a white bloodcell.

For observation of the visual cells P, an SLO image, such as thatillustrated in FIG. 6B, is captured with the focus position set near theouter layer of the retina (B4 in FIG. 6A). Retinal vessels and branchedcapillaries run through the inner layers of the retina (B1 to B3 in FIG.6A). When an adaptive optics SLO image is acquired with the focusposition set in an inner layer of the retina, it is possible to directlyobserve retinal vascular walls.

However, in a confocal image of the inner layer of the retina, strongnoise signals caused by reflection of light from a nerve fiber layer maymake it difficult to observe a vascular wall and detect wall boundaries.

Accordingly, a method of observing a non-confocal image has begun to beused in recent years. The non-confocal image is obtained by acquiringscattered light by varying the diameter, shape, and position of apinhole in front of a light receiving portion. A large focus depth ofthe non-confocal image facilitates observation of an object havingprotrusions and recesses in the depth direction, such as a vessel. Also,since light reflected from the nerve fiber layer is not easily directlyreceived, it is possible to achieve noise reduction.

A retinal artery is a small artery (arteriole) with a vessel diameter ofabout 10 μm to 100 μm. The wall of the retinal artery includes anintima, a media, and an adventitia. The media is formed by smooth-musclecells, and runs in a coil-like manner in the circumferential directionof the vessel. For example, if hypertension causes increased pressure onthe retinal arterial wall, the smooth muscle contracts and the wallthickness increases. At this point, the shape of the retinal arterialwall can be restored when the blood pressure is lowered, for example, bytaking a blood pressure lowering drug. However, if the hypertension isleft untreated for a long time, the smooth-muscle cells forming themedia become necrotic, and fibrous thickening of the media andadventitia leads to an increase in wall thickness. At this point,organic (irreversible) damage already develops in the retinal arterialwall, and continuous treatment is required to prevent worsening of thearteriolar damage.

A technique for measuring retinal vessel diameters is disclosed in“Computer algorithms for the automated measurement of retinal arteriolardiameters” by Chapman et al., published in Br J Ophthalmol, Vol. 85, No.1, pp. 74 to 79, 2001. In this technique, a luminance profile generatedon a line segment substantially perpendicular to the running of aretinal vessel in an SLO image is linearly approximated for each smallwindow. Then, positions corresponding to the maximum and minimum valuesof the slope of the resulting regression line are acquired as retinalvessel boundaries to measure the retinal vessel diameter. Additionally,a technique for semi-automatically extracting retinal vascular wallboundaries in an adaptive optics fundus camera image using a variablegeometry model is disclosed in “Morphometric analysis of small arteriesin the human retina using adaptive optics imaging: relationship withblood pressure and focal vascular changes” by Koch et al., published inJournal of Hypertension, Vol. 32, No. 4, pp. 890 to 898, 2014.

SUMMARY OF THE INVENTION

An image processing apparatus according to an aspect of the presentinvention includes a processing unit configured to perform, on aluminance distribution acquired in at least one direction crossing avessel region in a fundus image of a subject's eye, a first smoothingoperation for each first size and a second smoothing operation for eachsecond size smaller than the first size; a first identifying unitconfigured to identify a position of a vascular wall in the fundus imageon the basis of the luminance distribution obtained by performing thefirst smoothing operation; and a second identifying unit configured toidentify positions of inner and outer boundaries of the vascular wall inthe fundus image on the basis of the luminance distribution obtained byperforming the second smoothing operation.

An image processing method according to another aspect of the presentinvention includes a processing step of performing, on a luminancedistribution acquired in at least one direction crossing a vessel regionin a fundus image of a subject's eye, a first smoothing operation foreach first size and a second smoothing operation for each second sizesmaller than the first size; a first identifying step of identifying aposition of a vascular wall in the fundus image on the basis of theluminance distribution obtained by performing the first smoothingoperation; and a second identifying step of identifying positions ofinner and outer boundaries of the vascular wall in the fundus image onthe basis of the luminance distribution obtained by performing thesecond smoothing operation.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a functional configuration of animage processing apparatus according to a first embodiment of thepresent invention.

FIG. 2 is a block diagram illustrating a configuration of a systemincluding the image processing apparatus according to an embodiment ofthe present invention.

FIGS. 3A to 3H illustrate an overall configuration of an SLO imagingapparatus according to an embodiment of the present invention.

FIG. 4 is a block diagram illustrating a hardware configuration of acomputer that includes hardware components corresponding to a storageunit and an image processor, holds other units as software components,and executes the software.

FIGS. 5A and 5B are each a flowchart illustrating a process executed bythe image processing apparatus according to an embodiment of the presentinvention.

FIGS. 6A to 6M illustrate image processing in embodiments of the presentinvention.

FIG. 7 illustrates image processing in embodiments of the presentinvention.

FIGS. 8A to 8D are flowcharts illustrating details of operationsexecuted in steps S530 and S540 according to the first embodiment of thepresent invention.

FIGS. 9A and 9B each illustrate data displayed in step S560 according toan embodiment of the present invention.

FIG. 10 is a block diagram illustrating a functional configuration of animage processing apparatus according to a second embodiment of thepresent invention.

FIGS. 11A and 11B are flowcharts illustrating details of operationsexecuted in steps S530 and S540 according to the second embodiment ofthe present invention.

FIG. 12 is a block diagram illustrating a functional configuration of animage processing apparatus according to a third embodiment of thepresent invention.

FIGS. 13A to 13D are flowcharts illustrating details of operationsexecuted in steps S531 and S541 according to the third embodiment of thepresent invention.

DESCRIPTION OF THE EMBODIMENTS

Of all the arterioles in the body, the retinal arteries of the subject'seye are only tissues that are directly observable. For diagnosis ofdiabetes or hypertension, it is expected to accurately measure thethickness of the vascular walls of retinal arteries as a way ofdetecting the presence or degree of change in arterioles in the body.For the accurate measurement, it is required to easily and accuratelyidentify the positions of the inner and outer boundaries of the vascularwalls of retinal arteries.

An embodiment of the present invention provides a technique for easilyand accurately identifying the positions of the inner and outerboundaries of a vascular wall in a fundus image of a subject's eye.

An image processing apparatus according to an embodiment includes aprocessing unit (e.g., smoothing and differentiating unit 132)configured to perform, on a luminance distribution acquired in at leastone direction crossing a vessel region in a fundus image of a subject'seye, a first smoothing operation for each first size and a secondsmoothing operation for each second size smaller than the first size.The image processing apparatus according to the embodiment also includesa first identifying unit (e.g., approximate feature identifying unit1331) configured to identify a position of a vascular wall in the fundusimage on the basis of the luminance distribution obtained by performingthe first smoothing operation. The image processing apparatus accordingto the embodiment also includes a second identifying unit (e.g., wallfeature identifying unit 1332) configured to identify positions of innerand outer boundaries of the vascular wall in the fundus image on thebasis of the luminance distribution obtained by performing the secondsmoothing operation. With this configuration, the positions of the innerand outer boundaries of the vascular wall in the fundus image of thesubject's eye can be easily and accurately identified. The presentinvention may be configured in any manner as long as a smoothingoperation is performed on a luminance distribution acquired in at leastone direction. For example, the smoothing operation may be performed onthe entire fundus image or the entire vessel region.

An image processing apparatus according to another embodiment includes aprocessing unit (e.g., smoothing and differentiating unit 132)configured to perform, on a luminance distribution acquired in at leastone direction crossing a vessel region in a fundus image of a subject'seye, a smoothing operation with a first size and a second size smallerthan the first size. The image processing apparatus according to theembodiment also includes a first identifying unit (e.g., approximatefeature identifying unit 1331) configured to identify, as positions of afirst vascular wall and a second vascular wall, a plurality of featurepoints in the luminance distribution obtained by performing thesmoothing operation with the first size. The image processing apparatusaccording to the embodiment also includes a second identifying unit(e.g., wall feature identifying unit 1332) configured to identify, aspositions of inner and outer boundaries of the first vascular wall andpositions of inner and outer boundaries of the second vascular wall, aplurality of feature points near the identified positions of the firstand second vascular walls in the luminance distribution obtained byperforming the smoothing operation with the second size. With thisconfiguration, the positions of the inner and outer boundaries of thevascular wall in the fundus image of the subject's eye can be easily andaccurately identified. In related art, the operator manually measuresthe thicknesses of vascular walls and membranes of retinal arteries inthe fundus image of the subject's eye (e.g., an image of the fundus ofthe subject's eye captured by an SLO apparatus to which an adaptiveoptics technique is applied). Therefore, the measurement is complex andless reproducible due to measurement errors caused by the operator. As asolution to this, in an image depicting vascular walls of the subject'seye and membranes and cells forming the vascular walls, wall boundariesand membrane boundaries may be automatically identified and thethicknesses of the vascular walls and membranes may be automaticallymeasured. Accordingly, the image processing apparatus may furtherinclude a measuring unit configured to measure at least one of avascular wall thickness, a vessel inner diameter, and a vessel outerdiameter using the positions of the outer and inner boundaries of thefirst vascular wall and the positions of the outer and inner boundariesof the second vascular wall.

An image processing apparatus according to another embodiment includes afirst identifying unit (e.g., approximate feature identifying unit 1331)configured to identify a position of a vascular wall in a fundus imageof a subject's eye on the basis of a luminance distribution acquired inat least one direction crossing a vessel region in the fundus image. Theimage processing apparatus according to the embodiment also includes asecond identifying unit (e.g., wall feature identifying unit 1332)configured to identify a plurality of feature points near the identifiedposition of the vascular wall in the luminance distribution as positionsof inner and outer boundaries of the vascular wall in the fundus image.With this configuration, the positions of the inner and outer boundariesof the vascular wall in the fundus image of the subject's eye can beeasily and accurately identified. The document by Chapman et al.mentioned above discloses a technique of measuring a retinal vesseldiameter by acquiring retinal vessel boundaries in an SLO image using asliding linear regression filter. However, this document does notdescribe the technique of detecting the wall and membrane boundaries ofa vessel in an adaptive optics SLO image depicting vascular walls andmembranes, or the technique of automatically measuring the thicknessesof vascular walls and membranes. Also, the document by Koch et al.mentioned above discloses a technique of semi-automatically measuringthe thickness of a retinal arterial wall by detecting retinal vascularwall boundaries in an adaptive optics fundus camera image using avariable geometry model. However, since the adaptive optics funduscamera image cannot depict vein walls and membranes forming the arteryand vein walls, the document by Koch et al. does not describe thetechnique of detecting vein walls and membrane boundaries of arteriesand veins, or the technique of measuring the thicknesses of vein wallsand the thicknesses of vessel membranes of arteries and veins.

Embodiments of an image processing apparatus and an image processingmethod according to the present invention will now be described indetail with reference to the attached drawings. Note that the presentinvention is not limited to the embodiments described below.

First Embodiment

An image processing apparatus according to the present embodimentidentifies wall boundaries in one of images obtained by smoothing aretinal vascular wall image at different scales (sizes) anddifferentiating the resulting images, the retinal vascular wall imagebeing captured using an SLO apparatus configured to acquire confocal andnon-confocal images, on the basis of feature points on a luminanceprofile (luminance distribution) acquired in a direction crossing theretinal vessel.

Specifically, a non-confocal image of retinal vascular walls capturedusing the SLO apparatus configured to simultaneously acquire confocaland non-confocal images is smoothed at two different scales (sizes) anddifferentiated. In the image smoothed at a large scale (first size) anddifferentiated, the approximate positions (representative positions) ofthe walls (first and second vascular walls) are identified on the basisof feature points on a luminance profile crossing the retinal vessel. Inthe image smoothed at a small scale (second size) and differentiated,wall boundary candidate points are acquired from feature points on aluminance profile crossing the retinal vessel, and connected in thevessel running direction after removal of outliers, so as to identifyvascular wall boundaries and measure the wall thicknesses. In thepresent specification, the term “smoothing and differentiatingoperation” includes the concept of “differentiation (which may be eithersubtraction or division) performed on a luminance distribution obtainedby smoothing”, and also includes the concept of “linear approximationperformed on a luminance distribution”. Differentiation is not anessential operation for the present invention, but will be described indetail later on.

(General Configuration)

FIG. 2 illustrates a configuration of a system including an imageprocessing apparatus 10 according to the present embodiment. Asillustrated in FIG. 2, the image processing apparatus 10 is connected toan SLO imaging apparatus 20, a data server 40, and a pulse dataacquiring apparatus 50 through a local area network (LAN) 30 formed byan optical fiber, universal serial bus (USB), or IEEE 1394. The imageprocessing apparatus 10 may be connected to these apparatuses eitherdirectly or through an external network, such as the Internet.

The SLO imaging apparatus 20 is an apparatus configured to capture awide-field image Dl of the subject's eye, and a confocal image Dc and anon-confocal image Dn, which are high-magnification images, of thesubject's eye. The SLO imaging apparatus 20 transmits, to the imageprocessing apparatus 10 and the data server 40, the wide-field image Dl,the confocal image Dc, the non-confocal image Dn, and information offixation target positions Fl and Fcn used to capture the images.

The pulse data acquiring apparatus 50 is an apparatus configured toacquire biological signal data (pulse data) that autonomously changes.For example, the pulse data acquiring apparatus 50 is formed by asphygmograph or electrocardiograph. In response to an operation by anoperator (not shown), the pulse data acquiring apparatus 50 acquirespulse data Pi simultaneously with the acquisition of the wide-fieldimage Dl, the confocal image Dc, and the non-confocal image Dn. Theacquired pulse data Pi is transmitted to the image processing apparatus10 and the data server 40. The pulse data acquiring apparatus 50 may beconfigured to be directly connected to the SLO imaging apparatus 20.

When the images are acquired at different capturing positions, theacquired images are represented by Dli, Dcj, and Dnk, where i, j, and kare variables each representing a capturing position number as follows:i=1, 2, . . . , imax; j=1, 2, . . . , jmax; and k=1, 2, . . . , kmax.When the confocal image Dc and the non-confocal image Dn are eachacquired at different magnifications, the acquired images arerepresented by Dc1m, Dc2o, . . . (Dn1m, Dn2o, . . . ) in the descendingorder of magnification. Note that Dc1m (Dn1m) will be referred to as ahigh-magnification confocal (non-confocal) image, and Dc2o, . . . (Dn2o,. . . ) will be referred to as a medium-magnification confocal(non-confocal) image.

The data server 40 is configured to hold the wide-field image Dl, theconfocal image Dc, and the non-confocal image Dn of the subject's eye;condition data, such as the fixation target positions Fl and Fcn used tocapture the images; the pulse data Pi; and image features of thesubject's eye. As the image features of the subject's eye, the presentinvention deals with those related to retinal vessels, retinal vascularwalls, and membranes and wall cells forming the vascular walls. Thewide-field image Dl, the confocal image Dc, and the non-confocal imageDn output from the SLO imaging apparatus 20, the fixation targetpositions Fl and Fcn used to capture the images, the pulse data Pi, andthe image features of the subject's eye output from the image processingapparatus 10 are stored in the data server 40. In response to a requestfrom the image processing apparatus 10, the data server 40 transmits thewide-field image Dl, the confocal image Dc, the non-confocal image Dn,the pulse data Pi, and the image features of the subject's eye to theimage processing apparatus 10.

A functional configuration of the image processing apparatus 10according to the present embodiment will now be described with referenceto FIG. 1. FIG. 1 is a block diagram illustrating a functionalconfiguration of the image processing apparatus 10. The image processingapparatus 10 includes an image acquiring section 110, a storage unit120, an image processor 130, and an instruction acquiring unit 140.

The image acquiring section 110 includes a confocal data acquiring unit111, a non-confocal data acquiring unit 112, and a pulse data acquiringunit 113. The image processor 130 includes a luminance distributionacquiring unit 131, a smoothing and differentiating unit 132, anidentifying section 133, a connecting unit 134, a measuring unit 135,and a display control unit 136. The identifying section 133 includes anapproximate feature identifying unit 1331, which is an example of afirst identifying unit, and a wall feature identifying unit 1332, whichis an example of a second identifying unit.

The SLO imaging apparatus 20 to which adaptive optics is applied will bedescribed with reference to FIGS. 3A and 3B. The SLO imaging apparatus20 includes a super-luminescent diode (SLD) 201, a Shack-Hartmannwavefront sensor 206, an adaptive optics system 204, first and secondbeam splitters 202 and 203, an X-Y scanning mirror 205, a focus lens209, an aperture 210, an optical sensor 211, an image forming unit 212,and an output unit 213.

Light emitted from the SLD 201 serving as a light source is reflected bythe fundus. Part of the light is input to the Shack-Hartmann wavefrontsensor 206 through the second beam splitter 203, and the remaining partof the light is input to the optical sensor 211 through the first beamsplitter 202. The Shack-Hartmann wavefront sensor 206 is a device formeasuring aberration of the eye. The Shack-Hartmann wavefront sensor 206includes a lens array 207 and a charge-coupled device (CCD) 208connected to the lens array 207. When incident light passes through thelens array 207, a group of luminous points appears on the CCD 208, andwavefront aberration is measured on the basis of positional deviation ofthe luminous points projected on the CCD 208. On the basis of thewavefront aberration measured by the Shack-Hartmann wavefront sensor206, the adaptive optics system 204 drives an aberration correctingdevice (e.g., variable shape mirror or spatial optical phase modulator)to correct the aberration. The aberration-corrected light passes throughthe focus lens 209 and the aperture 210, and is received by the opticalsensor 211. A scanning position on the fundus can be controlled bymoving the X-Y scanning mirror 205. The optical sensor 211 acquires dataof an image target region and time (i.e., frame rate×number of frames)specified in advance by the operator. The optical sensor 211 transmitsthe data to the image forming unit 212, which corrects image distortioncaused by variation in scanning speed and also corrects luminance valuesto form image data (moving image or still image). The output unit 213outputs the image data formed by the image forming unit 212.

In the SLO imaging apparatus 20, the aperture 210 and the optical sensor211 in FIG. 3A may have any configuration as long as the confocal imageDc and the non-confocal image Dn can be acquired. In the presentembodiment, the aperture 210 is formed by a light-shielding portion210-1 (see FIGS. 3B and 3E), and the optical sensor 211 is formed byoptical sensors 211-1, 211-2, and 211-3 (see FIG. 3B). Referring to FIG.3B, part of return light (reflected light or scattered light) incidenton the light-shielding portion 210-1 disposed in the image forming planeis reflected and enters the optical sensor 211-1. The light-shieldingportion 210-1 will now be described with reference to FIG. 3E. Thelight-shielding portion 210-1 is formed by transmission regions 210-1-2and 210-1-3, a light-shielding region (not shown), and a reflectionregion 210-1-1. The light-shielding portion 210-1 is disposed such thatits center is located in the center of the optical axis of the returnlight. When the light-shielding portion 210-1 is positioned at an anglewith respect to the optical axis of the return light, thelight-shielding portion 210-1 has an elliptical pattern, which iscircular as viewed in the direction of the optical axis. Light split bythe light-shielding portion 210-1 enters the optical sensor 211-1. Lightpassing through the transmission regions 210-1-2 and 210-1-3 of thelight-shielding portion 210-1 is split by a prism 210-2 disposed in theimage forming plane, and enters the optical sensors 211-2 and 211-3 asillustrated in FIG. 3B.

A voltage signal obtained by each optical sensor is converted to adigital value by an analog-to-digital (AD) board in the image formingunit 212, and then converted to a two-dimensional image. An imagegenerated on the basis of the light incident on the optical sensor 211-1is a confocal image that focuses on a specific narrow range. An imagegenerated on the basis of the light incident on each of the opticalsensors 211-2 and 211-3 is a non-confocal image that focuses on a widerange.

A non-confocal signal may be divided in other ways. For example, asillustrated in FIG. 3F, a non-confocal signal may be divided into fourand received in transmission regions 210-1-4 to 210-1-7. Also, aconfocal signal and a non-confocal signal may be received in other ways.For example, the diameter and position of the aperture 210 (opening) maybe made variable, so that the aperture 210 can be adjusted to receiveeither a confocal signal as illustrated in FIG. 3C or a non-confocalsignal as illustrated in FIG. 3D. The diameter and the amount ofmovement of the opening may be set to any values. For example, thediameter of the opening can be set to about 1 airy disc diameter (ADD)in FIG. 3C, whereas the diameter of the opening can be set to about 10ADD and the amount of movement of the opening can be set to about 6 ADDin FIG. 3D. Alternatively, as illustrated in FIGS. 3G and 3H, aplurality of non-confocal signals may be received at substantially thesame time in a transmission region 210-1-8 or 210-1-9.

Since there are two types of non-confocal signals in the presentembodiment, a non-confocal image on one side (R channel image) isrepresented by Dnr and a non-confocal image on the other side (L channelimage) is represented by Dnl.

The non-confocal image Dn refers to both the R channel image Dnr and theL channel image Dnl.

In the configuration of FIG. 3A, if the swing angle of a scanningoptical system is increased and the adaptive optics system 204 isinstructed not to perform aberration correction, the SLO imagingapparatus 20 can also operate as a normal SLO apparatus and can acquirea wide-field image.

Hereinafter, an image with a magnification lower than thehigh-magnification images Dc and Dn and lowest among images acquired bythe image acquiring section 110 will be referred to as a wide-fieldimage Dl (Dlc, Dln). This means that the wide-field image Dl may be anSLO image to which adaptive optics is applied, or may be a simple SLOimage.

A hardware configuration of the image processing apparatus 10 will nowbe described with reference to FIG. 4. As illustrated in FIG. 4, theimage processing apparatus 10 includes a central processing unit (CPU)301, a memory (RAM) 302, a control memory (ROM) 303, an external storagedevice 304, a monitor 305, a keyboard 306, a mouse 307, and an interface308. Control programs for implementing the image processing function ofthe present embodiment and data used to execute the control programs arestored in the external storage device 304. Under the control of the CPU301, the control programs and data are appropriately loaded into the RAM302 through a bus 309 and executed by the CPU 301, and then serve as thefollowing units.

The functions of blocks forming the image processing apparatus 10 willbe described in relation to an execution procedure of the imageprocessing apparatus 10 illustrated in the flowchart of FIG. 5A.

(Step S510)

The image acquiring section 110 makes a request to the SLO imagingapparatus 20 to acquire the wide-field image Dl and thehigh-magnification images (confocal image Dcj, and non-confocal imagesDnrk and Dnlk). The image acquiring section 110 also makes a request tothe SLO imaging apparatus 20 to acquire the fixation target positions Fland Fcn corresponding to these images. In response to the acquisitionrequest, the SLO imaging apparatus 20 acquires the wide-field image Dl,the confocal image Dcj, the non-confocal images Dnrk and Dnlk, thecorresponding attribute data, and the fixation target positions Fl andFcn and transmits them to the image acquiring section 110. The imageacquiring section 110 receives the wide-field image Dl, the confocalimage Dcj, the non-confocal images Dnrk and Dnlk, and the fixationtarget positions Fl and Fcn from the SLO imaging apparatus 20 throughthe LAN 30, and stores them in the storage unit 120.

The pulse data acquiring unit 113 makes a request to the pulse dataacquiring apparatus 50 to acquire the pulse data Pi related to abiological signal. In the present embodiment, a sphygmograph serving asthe pulse data acquiring apparatus 50 acquires pulse wave data as thepulse data Pi from an earlobe of the subject. Here, the pulse data Pi isexpressed as a sequence of points having the time of acquisition on oneaxis and a pulse wave signal value measured by the sphygmograph on theother axis. The pulse data acquiring apparatus 50 acquires and transmitsthe corresponding pulse data Pi in response to the acquisition request.The pulse data acquiring unit 113 receives the pulse data Pi from thepulse data acquiring apparatus 50 through the LAN 30. The pulse dataacquiring unit 113 stores the received pulse data Pi in the storage unit120.

The confocal data acquiring unit 111 or the non-confocal data acquiringunit 112 may start image acquisition in accordance with a specific phaseof the pulse data Pi acquired by the pulse data acquiring apparatus 50,or the acquisition of the pulse data Pi and the image acquisition may besimultaneously started immediately after the request for imageacquisition. In the present embodiment, the acquisition of the pulsedata Pi and the image acquisition are simultaneously started immediatelyafter the request for image acquisition.

The pulse data Pi for each image is acquired from the pulse dataacquiring unit 113, and extremal values of each pulse data Pi aredetected to calculate the cardiac beat cycle and the relative cardiaccycle. The relative cardiac cycle is a relative value expressed as afloating-point value ranging from 0 to 1, with the cardiac beat cyclebeing 1.

Examples of the confocal image Dc and the non-confocal image Dnr of aretinal vessel are illustrated in FIGS. 6C and 6D. In the confocal imageDc, reflection from a nerve fiber layer on the background is strong, andthis often makes registration difficult due to the background noise. Inthe non-confocal image Dnr on the R channel, the contrast of thevascular wall on the right side is higher. In the non-confocal image Dnlon the L channel (see FIG. 6E), the contrast of the vascular wall on theleft side is higher.

One of the following images (a) and (b), each obtained by arithmeticoperation between the R channel image and the L channel image, may beused as a non-confocal image to observe and measure the vascular walls:

(a) averaged image Dnr+l of the R channel image and the L channel image(see FIG. 6G); and

(b) split detector image Dns obtained by a difference enhancingoperation between non-confocal images ((L−R)/(R+L)) (see FIG. 6F).

High-magnification images acquired at any position may be used. Forexample, the present invention includes the case of using imagesacquired around the optic disk, or images acquired along the retinalvessel arcade.

(Step S520)

The image processor 130 performs inter-frame registration of theacquired images. Next, the image processor 130 determines exceptionframes on the basis of the luminance value and noise of each frame andthe amount of displacement from a reference frame.

The image processor 130 first performs inter-frame registration in thewide-field image Dl and the confocal image Dc, and applies parametervalues for the inter-frame registration to each of the non-confocalimages Dnr and Dnl.

Examples of the technique for the inter-frame registration include thefollowing.

(i) The image processor 130 sets a reference frame serving as areference for the registration. In the present embodiment, a frame withthe smallest frame number is defined as a reference frame. The methodfor setting a reference frame is not limited to this, and any method maybe used.

(ii) The image processor 130 establishes rough positionalcorrespondences (i.e., performs rough registration) between frames.Although any registration technique can be used, the present embodimentperforms rough registration using a correlation coefficient as aninter-image similarity evaluation function and also using affinetransformation as a coordinate transformation technique.

(iii) The image processor 130 performs precise registration on the basisof data on rough positional correspondences between frames.

The present embodiment performs precise registration between frames of amoving image obtained after the rough registration in (ii) using afree-form deformation (FFD) technique, which is a kind of non-rigidregistration techniques.

The technique for precise registration is not limited to this, and anyregistration technique may be used.

In the present embodiment, registration parameters obtained byperforming inter-frame registration of the confocal image Dc are used asinter-frame registration parameters for the non-confocal image Dn, butthe present invention is not limited to this. For example, the presentinvention also includes the case of using registration parametersobtained by inter-frame registration of the non-confocal image Dn(including Dnr, Dnl, and an image obtained by arithmetic operationbetween Dnr and Dnl) as inter-frame registration parameters for theconfocal image Dc.

Next, the image processor 130 performs registration between thewide-field image Dl and the high-magnification image Dcj, and determinesthe relative position of the high-magnification image Dcj on thewide-field image Dl.

From the storage unit 120, the image processor 130 acquires the fixationtarget position Fcn used to capture the high-magnification image Dcj,and uses the fixation target position Fcn as an initial search point forthe registration parameters for registration between the wide-fieldimage Dl and the high-magnification image Dcj. The image processor 130performs registration between the wide-field image Dl and thehigh-magnification image Dcj while varying the combination of theparameter values.

A combination of registration parameter values having the highestsimilarity between the wide-field image Dl and the high-magnificationimage Dcj is determined as the relative position of thehigh-magnification image Dcj with respect to the wide-field image Dl.The registration technique is not limited to this, and any registrationtechnique may be used.

If a medium-magnification image has been acquired in step S510,registration of images is performed in the ascending order ofmagnification. For example, if the high-magnification confocal imageDc1m and the medium-magnification confocal image Dc2o have beenacquired, registration between the wide-field image Dl and themedium-magnification confocal image Dc2o is performed first, and thenregistration between the medium-magnification confocal image Dc2o andthe high-magnification confocal image Dc1m is performed.

Additionally, image-combining parameter values determined for thewide-field image Dl and the confocal image Dcj are also applied tocombining the non-confocal images Dnrk and Dnlk. The relative positionof each of the non-confocal images Dnrk and Dnlk on the wide-field imageDl is determined.

(Step S530)

The approximate feature identifying unit 1331, which is an example ofthe first identifying unit, identifies the approximate position of eachvascular wall in the following procedure.

(i) After frame averaging of the non-confocal moving images obtainedafter the inter-frame registration in step S520, the resulting image issmoothed at two different scales (sizes). The smoothed images aredifferentiated to generate smoothed and differentiated images. Thesmoothing is performed on the entire image in the present embodiment,but may be performed on a luminance distribution in at least onedirection crossing a vessel region in a fundus image. In the fundusimage, differentiation may be performed in two directions perpendicularto each other. However, when a luminance distribution in a directioncrossing a vessel region is differentiated, the differentiation may beperformed along the direction crossing the vessel region.

(ii) Of the smoothed and differentiated images generated in (i), theimage obtained by differentiating the image smoothed at a small scale issubjected to a morphological filter to detect the center line of theretinal vessel.

(iii) Of the smoothed and differentiated images generated in (i), theimage obtained by performing a first smoothing operation at a largescale (i.e., with a large filter size or large filter coefficient),which is defined as a first size, is subjected to a firstdifferentiating operation. At each position on the vessel center line inthe smoothed and differentiated image obtained by performing the firstdifferentiating operation, a luminance profile (luminance distribution)is acquired on a line segment substantially perpendicular to the vesselcenter line.

(iv) On the luminance profile generated in (iii), the maximum andminimum values are detected in the rightward and leftward directions,respectively, to identify the approximate positions of walls. Aftergrouping approximate position candidate points for each of the left walland the right wall, which are a first vascular wall and a secondvascular wall, respectively, the distances from the approximate positioncandidate points of each wall to the vessel center line are calculated.A predetermined percentage of the distance values at the top and bottomin each group are considered as outliers, and the approximate positioncandidate points having the outliers are removed. Then, the remainingapproximate position candidate points of each wall are interpolated inthe wall running direction. The technique for determining the outliersis not limited to that based on the distance values from the center lineas in the present embodiment, and any known technique may be used todetermine the outliers.

The process of identifying the approximate wall positions will bedescribed in detail later on in steps S810 to S860.

(Step S540)

The wall feature identifying unit 1332, which is an example of thesecond identifying unit, identifies avascular wall boundary positions inthe following procedure.

(i) Of the smoothed and differentiated images generated in (i) of stepS530, the image obtained by performing a second smoothing operation atthe small scale (i.e., with a small filter size or small filtercoefficient), which is defined as a second size, is subjected to asecond differentiating operation. At each position on the vessel centerline in the smoothed and differentiated image obtained by performing thesecond differentiating operation, a luminance profile (luminancedistribution) is acquired on a line segment substantially perpendicularto the vessel center line. Also, the approximate wall positionsidentified in (iv) of step S530 are acquired.

(ii) On the luminance profile acquired in (i), two local maximum pointsnear the approximate wall position on the right side and two localminimum points near the approximate wall position on the left side aredetected, and the detected points are defined as wall boundary candidatepoints. The wall boundary candidate points are grouped into four groups:an outer boundary of the left wall, an inner boundary of the left wall,an outer boundary of the right wall, and an inner boundary of the rightwall. Then, the first-order moment (i.e., “distance from the centerline”×“luminance value”) is calculated for each wall boundary candidatepoint. Here, a luminance value in the non-confocal image (R+L image) isreferenced as the luminance value. The technique for determiningoutliers is not limited to that based on the first-order moment as inthe present embodiment, and any known technique may be used to determinethe outliers. In each group, a predetermined percentage of the momentvalues at the top and bottom are considered as outliers, and the wallboundary candidate points having the outliers are removed. Then, theremaining wall boundary candidate points are interpolated in the wallrunning direction to identify the wall boundary.

The process of identifying the wall boundaries will be described indetail later on in steps S811 to S841.

(Step S550)

On the basis of the positions of the vascular wall boundaries identifiedin step S540, the measuring unit 135 measures the distribution of wallthickness along the running of the vessel, and the distribution of indexvalue related to the wall thickness.

Specifically, the measuring unit 135 calculates the wall thickness ofthe detected wall, the inner and outer diameters of the vessel, and theindex value (wall-to-lumen ratio (WLR)=(vessel outer diameter−vesselinner diameter)/(vessel inner diameter)) related to the wall thickness,and then determines the average value, standard deviation, and maximumand minimum values. These statistical values may be calculated not onlyfor the entire image, but also for each branch vessel. Also, thestatistical values of the wall thickness and the index value related tothe wall thickness may be calculated for each side (i.e., right or leftside in the vessel running direction) within the branch vessel, or maybe calculated for each small region.

The index value related to the vascular wall thickness is not limited tothis, and may be calculated by arithmetic operation of wall thicknessvalues calculated for walls on both the right and left sides. Forexample, the wall thickness ratio between the right and left sides, asviewed in the vessel running direction, may be calculated. Since wallcells forming the majority of the vascular wall run in a coil-likemanner, abnormalities in wall thickness are likely to occur on bothsides. Therefore, the wall thickness ratio is used as an index forreliability of the measured wall thickness values.

(Step S560)

The display control unit 136 displays the acquired images, the positionsof detected wall boundaries, and the measurement result (the wallthickness and the index value related thereto) on the monitor 305. Inthe present embodiment, the display control unit 136 displays thefollowing (i) to (iv) on the monitor 305:

(i) non-confocal moving image (I1 in FIG. 9A),

-   -   image obtained by selecting frames corresponding to a specific        phase of a pulse wave and averaging the frames (I2 in FIG. 9A),        and    -   image extracting the lumen of the vessel for comparison (I3 in        FIG. 9A);

(ii) detected wall boundary positions (Bv in FIG. 9A);

(iii) graph showing the wall thickness measured along the running of thevascular wall or the index value related to the wall thickness (G1 inFIG. 9A); and

(iv) map showing the distribution of the wall thickness calculated foreach small region or the index value related to the wall thickness (FIG.9B).

For (iv), the display control unit 136 associates the calculated valueswith a color bar and displays them in color.

(Step S570)

The instruction acquiring unit 140 externally acquires an instruction asto whether to store the images acquired in step S510 and data measuredin step S550 (i.e., wall boundary positions and wall thickness values inthe non-confocal image Dnk) in the data server 40. For example, thisinstruction is input by the operator with the keyboard 306 or the mouse307. If an instruction to store the images and data described above isacquired, the process proceeds to step S580, and if not, the processproceeds to step S590.

(Step S580)

The image processor 130 associates the examination date and informationidentifying the subject's eye with the images and themeasurement-related data (determined to be stored in step S570), andtransmits them to the data server 40.

(Step S590)

The instruction acquiring unit 140 externally acquires an instruction asto whether to end the process performed by the image processingapparatus 10 for the non-confocal image Dnk. This instruction is inputby the operator with the keyboard 306 or the mouse 307. If aninstruction to end the process is acquired, the process ends here. If aninstruction to continue the process is acquired, the process returns tostep S510, where the operation on the next subject's eye (or the samesubject's eye) is performed.

Details of the operation performed in step S530 of FIG. 5A will bedescribed with reference to FIGS. 6A to 6M, FIG. 7, and the flowchart ofFIG. 8A.

(Step S810)

The smoothing and differentiating unit 132 performs smoothing, atmultiple scales, on the non-confocal moving images obtained after theinter-frame registration. Any known smoothing operation is applicablehere. In the present embodiment, after frame averaging of non-confocalmoving images Dr+l obtained by inter-frame registration, the resultingimage is subjected to a mean filter with filter sizes of 4 and 10.

(Step S820)

The smoothing and differentiating unit 132 performs differentiation onthe smoothed images generated in step S810. Although any knowndifferentiating operation is applicable, a differential edge detectionoperator is used in the present embodiment. As illustrated in FIG. 61,the resulting smoothed and differentiated images are similar to an imageobtained by smoothing the split detector image Dns (see FIG. 6F).

(Step S830)

The luminance distribution acquiring unit 131 applies a morphologicalfilter to the image smoothed with a smaller filter size in step S810, soas to detect the center line of the retinal artery. In the presentembodiment, the luminance distribution acquiring unit 131 uses a top-hatfilter to detect a narrow-width, high-luminance region corresponding toreflection from the vascular wall. Then, the luminance distributionacquiring unit 131 performs thinning on the high-luminance region todetect the vessel center line. The method for detecting the vesselcenter line is not limited to this, and any known detecting method maybe used.

(Step S840)

At each position on the vessel center line in the image (see FIG. 61)differentiated after being smoothed at a large scale, the luminancedistribution acquiring unit 131 generates a luminance profile Gpri1 (702in FIG. 7, i=0, 1, 2, . . . , N) along a line segment perpendicular tothe vessel center line.

Additionally, the approximate feature identifying unit 1331 searchesluminance values on the luminance profile Gpri1 (702 in FIG. 7) toacquire a minimum value Gmin and a maximum value Gmax on the left andright sides, respectively, of a center line position G0, therebyidentifying approximate wall position candidates.

(Step S850)

The image processor 130 groups the approximate wall position candidatesidentified on the luminance profiles in step S840 into the following twogroups:

(i) approximate position candidate group for the left wall, and

(ii) approximate position candidate group for the right wall.

Then, the image processor 130 determines outliers in each approximateposition candidate group and removes them. In the present embodiment, atop Tt1% and a bottom Tb1% in terms of the distance from the vesselcenter line are considered as outliers, and approximate positioncandidates having the corresponding distance values are removed.

(Step S860)

The connecting unit 134 interpolates the remaining approximate wallposition candidates in the vessel running direction and connects them toidentify the vascular wall boundary. Any known technique may be used forthe interpolation and connection. In the present embodiment, a naturalspline interpolation method is used to interpolate and connect theapproximate wall position candidates.

Details of the operation performed in step S540 of FIG. 5A will now bedescribed with reference to FIGS. 6A to 6M, FIG. 7, and the flowchart ofFIG. 8B.

(Step S811)

At each position on the vessel center line in the image differentiatedafter being smoothed at the small scale, a luminance profile Gpri2 (703in FIG. 7, i=0, 1, 2, . . . , N) along a line segment perpendicular tothe vessel center line and approximate wall positions Gmin and Gmax areacquired.

(Step S821)

In the luminance profile Gpri2 acquired in step S811, the wall featureidentifying unit 1332 selects two local minimum values near theapproximate wall position Gmin and two local maximum values near theapproximate wall position Gmax to identify wall boundary candidatepositions.

(Step S831)

The image processor 130 groups the wall boundary candidates identifiedon the luminance profiles in step S821 into the following four groups:

(i) outer boundary candidate group for the left wall,

(ii) inner boundary candidate group for the left wall,

(iii) outer boundary candidate group for the right wall, and

(iv) inner boundary candidate group for the right wall.

Then, the image processor 130 determines outliers in each boundarycandidate group and removes them. In the present embodiment, thefirst-order moment (i.e., “luminance value”×“distance from the vesselcenter line”) is calculated for each boundary candidate group. Then, atop Tt2% and a bottom Tb2% are considered as outliers, and wall boundarycandidates having the corresponding moment values are removed. Here, aluminance value in the non-confocal image (R+L image) is referenced asthe luminance value.

(Step S841)

The connecting unit 134 interpolates the remaining wall boundarycandidate points for each boundary candidate group in the vessel runningdirection and connects them to identify the vascular wall boundary. Anyknown technique may be used for the interpolation and connection. In thepresent embodiment, a natural spline interpolation method is used tointerpolate and connect the wall boundary candidate points.

For smoothing and differentiating a luminance profile, the presentembodiment applies a smoothing filter and a differentiating filter to anon-confocal image to acquire a luminance profile on the smoothed anddifferentiated image. However, the present invention is not limited tothis. For example, in the procedures illustrated in FIGS. 8C and 8D, aluminance profile is acquired in a direction crossing a vessel in anon-confocal image. The present invention also includes the case ofgenerating a smoothed and differentiated luminance profile by repeatingthe process which involves linearly approximating the luminance profilein each small window and outputting the slope of the regression linewhile moving the small window at regular intervals. In this case, onlyluminance values on the luminance profile, instead of the entire image,are smoothed and differentiated. A first linear approximation may beperformed in a window of large size corresponding to the first scale,and a second linear approximation may be performed in a window of smallsize corresponding to the second scale.

Although the present embodiment identifies vascular wall boundaries inthe non-confocal moving images Dr+l, the present invention is notlimited to this. For example, the present invention also includes thecase of identifying wall boundaries in the confocal image Dc by applyingthe image processing technique described in the present embodiment.

In the present embodiment, a single type of non-confocal image (R+Limage) is smoothed at different scales and differentiated, so as toidentify approximate vascular wall positions and vascular wallboundaries on the basis of feature points on a luminance profile in thesmoothed and differentiated image. However, the present invention is notlimited to this. For example, different types of non-confocal images (Rchannel image and L channel image) are differentiated after beingsmoothed at different scales. The approximate position and the wallboundaries of the vascular wall on the left side are identified byreferring to a luminance profile in the image obtained by smoothing anddifferentiating the L channel image, and the approximate position andthe wall boundaries of the vascular wall on the right side areidentified by referring to a luminance profile in the image obtained bysmoothing and differentiating the R channel image. This is also includedin the present invention.

The present embodiment deals with the case in which, when a moving imageobtained after registration is smoothed at different scales, thesmoothing is performed with different filter sizes after frameaveraging. However, the present invention is not limited to this. Forexample, the present invention also includes the case in which an imageobtained by frame averaging alone is used as an image smoothed at asmall scale, and an image obtained by in-plane smoothing after frameaveraging is used as an image smoothed at a large scale. The presentinvention also includes the case in which two-dimensional imagessmoothed at different scales are generated by applying athree-dimensional smoothing filter having different filter sizes in thein-plane direction to a moving image obtained after inter-frameregistration. In the case of performing frame averaging, framescorresponding to a specific phase of a pulse wave may be selected andaveraged, so as to prevent the positions of membrane boundaries formingthe vascular wall from being changed by the impact of cardiac beats.

Although the present embodiment deals with the case of automaticallyacquiring a vessel center line using a morphological filter, the presentinvention is not limited to this. For example, the present inventionalso includes the case of manually setting a vessel center line byacquiring, from the instruction acquiring unit 140, the position of thevessel center line specified by the operator using the keyboard 306 orthe mouse 307.

With the configuration described above, in images obtained by smoothinga retinal vascular wall image at different scales and differentiatingthe resulting images, the retinal vascular wall image being capturedusing an SLO apparatus configured to acquire confocal and non-confocalimages, the image processing apparatus 10 identifies wall boundaries onthe basis of feature points on luminance profiles acquired in adirection crossing the vessel.

Thus, a vascular wall region can be easily and robustly identified in animage of the subject's eye.

Second Embodiment

An image processing apparatus according to the present embodimentperforms the following processing on an image captured by an SLOapparatus configured to acquire a plurality of types of non-confocalimages. First, the image processing apparatus smoothes two types ofnon-confocal images at different scales. Additionally, the imageprocessing apparatus performs a subtraction operation between differenttypes of non-confocal images smoothed at a large scale to generate asplit detector image smoothed at a large scale. Also, the imageprocessing apparatus performs a subtraction operation between differenttypes of non-confocal images smoothed at a small scale to generate asplit detector image smoothed at a small scale. A description will nowbe given of the case where, by using the same technique as the firstembodiment, the approximate position of each vascular wall is identifiedin the split detector image smoothed at the large scale and vascularwall boundaries are identified in the split detector image smoothed atthe small scale.

The configuration of devices connected to the image processing apparatus10 according to the present embodiment will not be described here, as itis the same as that in the first embodiment.

FIG. 10 illustrates functional blocks of the image processing apparatus10 according to the present embodiment. The present embodiment differsfrom the first embodiment in that the image processor 130 includes asmoothing unit 137 and a non-confocal data operation unit 138, insteadof the smoothing and differentiating unit 132.

The image processing flow of the present embodiment is as illustrated inFIG. 5A, and steps other than steps S530 and S540 will not be describedas they are the same as those in the first embodiment.

(Step S530)

The approximate feature identifying unit 1331 identifies the approximateposition of each vascular wall in the following procedure.

(i) After frame averaging of the non-confocal moving images obtainedafter the inter-frame registration in step S520, the resulting image issmoothed at two different scales.

(ii) By performing a subtraction operation between different types ofnon-confocal images smoothed at a large scale and a subtractionoperation between different types of non-confocal images smoothed at asmall scale, two split detector images of different smoothing scales aregenerated.

(iii) Of the images smoothed in (i), the non-confocal image smoothed atthe small scale is subjected to a morphological filter to detect thecenter line of the retinal vessel.

(iv) At each position on the vessel center line in one of the imagesgenerated in (ii), the one being a split detector image smoothed at thelarge scale, a luminance profile is acquired on a line segmentsubstantially perpendicular to the vessel center line.

(v) On the luminance profile generated in (iv), the maximum and minimumvalues are detected in the rightward and leftward directions,respectively, to identify the approximate positions of walls. Aftergrouping the approximate position candidate points for each of the leftand right walls, the distances from the approximate position candidatepoints of each wall to the vessel center line are calculated. Apredetermined percentage of the distance values at the top and bottom ineach group are considered as outliers, and the approximate positioncandidate points having the outliers are removed. Then, the remainingapproximate position candidate points of each wall are interpolated inthe wall running direction. Any known technique may be used to determinethe outliers.

Details of the operation performed in step S530 of FIG. 5A will bedescribed with reference to FIG. 7 and the flowchart of FIG. 11A. Notethat steps S1130, S1150, and S1160 will not be described here, as theyare the same as the corresponding steps in the first embodiment.

(Step S1110)

The smoothing unit 137 performs smoothing, at multiple scales, on thenon-confocal moving images obtained after the inter-frame registration.Any known smoothing operation is applicable here. In the presentembodiment, after frame averaging of non-confocal moving images Dnr andDnl obtained by inter-frame registration, the resulting image issubjected to a Gaussian filter with filter sizes of 4 and 10.

(Step S1120)

The non-confocal data operation unit 138 performs a subtractionoperation between different types of non-confocal images smoothed instep S1110. In the present embodiment, a difference enhancing operationis performed using the R channel image and the L channel image smoothedat the same scale ((L−R)/(R+L)) to generate a smoothed split detectorimage (see FIG. 6F). The operation performed here is not limited tothis, and any known operation having the same effect as differentiationmay be applied. Instead of performing the subtraction operationdescribed above, the non-confocal data operation unit 138 may perform adivision operation between different types of smoothed non-confocalimages.

(Step S1140)

At each position on the vessel center line in the split detector image(see FIG. 6F) smoothed at the large scale, the luminance distributionacquiring unit 131 generates a luminance profile Spri1 (704 in FIG. 7,i=0, 1, 2, . . . , N) along a line segment perpendicular to the vesselcenter line.

The approximate feature identifying unit 1331 searches luminance valueson the luminance profile Spri1 (704 in FIG. 7) to acquire a minimumvalue Gmin and a maximum value Gmax on the left and right sides,respectively, of a center line position G0, thereby identifyingapproximate wall position candidates.

(Step S540)

The wall feature identifying unit 1332 identifies vascular wall boundarypositions in the following procedure.

(i) At each position on the vessel center line in the split detectorimage smoothed at the small scale in (i) of step S530, a luminanceprofile Spri2 (705 in FIG. 7) is acquired on a line segmentsubstantially perpendicular to the vessel center line. Also, theapproximate wall positions identified in (v) of step S530 are acquired.

(ii) On the luminance profile acquired in (i), two local maximum pointsnear the approximate wall position on the right side and two localminimum points near the approximate wall position on the left side aredetected, and the detected points are defined as wall boundary candidatepoints. The detected wall boundary candidate points are grouped intofour groups: an outer boundary of the left wall, an inner boundary ofthe left wall, an outer boundary of the right wall, and an innerboundary of the right wall. Then, the first-order moment (i.e.,“distance from the center line”×“luminance value”) is calculated foreach wall boundary candidate point. Here, a luminance value in thenon-confocal image (i.e., L channel image for the boundary candidate ofthe left wall, and R channel image for the boundary candidate of theright wall) is referenced as the luminance value. A predeterminedpercentage of the moment values at the top and bottom in each group areconsidered as outliers, and the wall boundary candidate points havingthe outliers are removed. Then, the remaining wall boundary candidatepoints are interpolated in the wall running direction to identify thewall boundary. Any known technique may be used to determine theoutliers.

Details of the operation performed in step S540 of FIG. 5A will bedescribed with reference to FIG. 7 and the flowchart of FIG. 11B. Notethat step S1141 will not be described here, as it is the same as thecorresponding step in the first embodiment.

(Step S1111)

At each position on the vessel center line in the split detector imagesmoothed at the small scale, a luminance profile Spri2 (705 in FIG. 7,i=0, 1, 2, . . . , N) along a line segment perpendicular to the vesselcenter line and approximate wall positions Gmin and Gmax are acquired.

(Step S1121)

In the luminance profile Spri2 acquired in step S1111, the wall featureidentifying unit 1332 selects two local minimum values near theapproximate wall position Gmin and two local maximum values near theapproximate wall position Gmax to identify wall boundary candidatepositions.

(Step S1131)

The image processor 130 groups the wall boundary candidates identifiedon the luminance profiles in step S1121 into the following four groups:

(i) outer boundary candidate group for the left wall,

(ii) inner boundary candidate group for the left wall,

(iii) outer boundary candidate group for the right wall, and

(iv) inner boundary candidate group for the right wall.

Then, the image processor 130 determines outliers in each boundarycandidate group and removes them. In the present embodiment, thefirst-order moment (i.e., “luminance value”×“distance from the vesselcenter line”) is calculated for each boundary candidate group. Then, atop Tt2% and a bottom Tb2% are considered as outliers, and wall boundarycandidates having the corresponding moment values are removed. Here, aluminance value in the non-confocal image (i.e., L channel image forboundary candidates for the left wall, and R channel image for boundarycandidates for the right wall) is referenced as the luminance value.

The present embodiment generates a smoothed and differentiated image byperforming a subtraction operation between different types ofnon-confocal images smoothed at the same scale, but the presentinvention is not limited to this. For example, the present inventionalso includes the case of generating a smoothed and differentiated imageby smoothing, at different scales, a split detector image generated byperforming a subtraction operation between different types ofnon-confocal images.

The present embodiment deals with the case in which, when a moving imageobtained after registration is smoothed at different scales, thesmoothing is performed with different filter sizes after frameaveraging. However, the present invention is not limited to this. Forexample, the present invention also includes the case in which an imageobtained by frame averaging alone is used as an image smoothed at asmall scale, and an image obtained by in-plane smoothing after frameaveraging is used as an image smoothed at a large scale. The presentinvention also includes the case in which two-dimensional imagessmoothed at different scales are generated by applying athree-dimensional smoothing filter having different filter sizes in thein-plane direction to a moving image obtained after inter-frameregistration. In the case of performing frame averaging, framescorresponding to a specific phase of a pulse wave may be selected andaveraged, so as to prevent the positions of membrane boundaries formingthe vascular wall from being changed by the impact of cardiac beats.

Although the present embodiment deals with the case of automaticallyacquiring a vessel center line using a morphological filter, the presentinvention is not limited to this. For example, the present inventionalso includes the case of manually setting a vessel center line byacquiring, from the instruction acquiring unit 140, the position of thevessel center line specified by the operator using the keyboard 306 orthe mouse 307.

With the configuration described above, the image processing apparatus10 captures a retinal vascular wall image using an SLO apparatusconfigured to acquire different types of non-confocal images, andgenerates a split detector image smoothed at a large scale and a splitdetector image smoothed at a small scale. The image processing apparatus10 identifies wall boundaries on the basis of feature points onluminance profiles acquired, in a direction crossing the retinal vessel,in the split detector image smoothed at the large scale and the splitdetector image smoothed at the small scale.

Thus, a vascular wall region can be easily and robustly identified in animage of the subject's eye.

Third Embodiment

In images obtained by smoothing each retinal vascular wall image atdifferent scales and differentiating the resulting images, the retinalvascular wall image being captured by an SLO apparatus configured toacquire a plurality of types of non-confocal images, an image processingapparatus according to the present embodiment identifies membraneboundaries of the vessel on the basis of feature points on luminanceprofiles acquired in a direction crossing the vessel.

Specifically, non-confocal images (R channel image and L channel image)of a retinal vascular wall captured using the SLO apparatus configuredto simultaneously acquire confocal and non-confocal images are smoothedat two different scales and differentiated. In the image smoothed at alarge scale and differentiated, the approximate position of the wall isidentified from feature points on a luminance profile crossing theretinal vessel. In the image smoothed at a small scale anddifferentiated, membrane boundary candidate points are acquired fromfeature points on a luminance profile crossing the retinal vessel. Afterremoval of outliers, the remaining membrane boundary candidate pointsare connected in the vessel running direction to identify the vesselmembrane boundaries and measure the vessel membrane thickness.

The configuration of devices connected to the image processing apparatus10 according to the present embodiment will not be described here, as itis the same as that in the first embodiment.

FIG. 12 illustrates functional blocks of the image processing apparatus10 according to the present embodiment. The present embodiment differsfrom the first embodiment in that the image processor 130 includes amembrane feature identifying unit 1333 instead of the wall featureidentifying unit 1332.

The image processing flow of the present embodiment is as illustrated inFIG. 5B, and steps other than steps S531, S541, S551, and S561 will notbe described as they are the same as the corresponding steps in thefirst embodiment.

(Step S531)

The approximate feature identifying unit 1331 identifies the approximateposition of each vascular wall in the following procedure.

(i) After frame averaging of the non-confocal images obtained after theinter-frame registration in step S521, the resulting image is smoothedat two different scales and the smoothed images are differentiated.

(ii) Of the images smoothed in (i), the non-confocal image smoothed at asmall scale is subjected to a morphological filter to detect the centerline of the retinal vessel.

(iii) At each position on the vessel center line in the imagedifferentiated after being smoothed at a large scale in (i), a luminanceprofile is acquired on a line segment substantially perpendicular to thevessel center line.

(iv) On the luminance profile generated in (iii), the maximum value isdetected in the rightward direction in the case of the R channel imageand the minimum value is detected in the leftward direction in the caseof the L channel image, so as to identify the approximate position ofeach wall. After grouping the approximate position candidate points foreach of the left and right walls, the distances from the approximateposition candidate points of each wall to the vessel center line arecalculated. A predetermined percentage of the distance values at the topand bottom in each group are considered as outliers, and the approximateposition candidate points having the outliers are removed. Then, theremaining approximate position candidate points of each wall areinterpolated in the wall running direction. Any known technique may beused to determine the outliers.

Details of the operation performed in step S531 of FIG. 5B will bedescribed with reference to FIG. 7 and the flowchart of FIG. 13A. Notethat steps S1330, S1350, and S1360 will not be described here, as theyare the same as the corresponding steps in the first embodiment.

(Step S1310)

The smoothing and differentiating unit 132 performs smoothing, atmultiple scales, on the non-confocal moving images (R channel image andL channel image) obtained after the inter-frame registration. Any knownsmoothing operation is applicable here. In the present embodiment, afterframe averaging of the non-confocal images Dnr and Dnl obtained afterinter-frame registration, the resulting image is subjected to a meanfilter with filter sizes of 4 and 10.

(Step S1320)

The smoothing and differentiating unit 132 performs differentiation onthe R channel image and the L channel image smoothed in step S1310 togenerate smoothed and differentiated images. Although any knowndifferentiating operation is applicable, a differential edge detectionoperator is used in the present embodiment.

(Step S1340)

At each position on the vessel center line in the R channel image andthe L channel image differentiated after being smoothed at the largescale, the luminance distribution acquiring unit 131 generates aluminance profile along a line segment perpendicular to the vesselcenter line. FIG. 6K illustrates a luminance profile generation positionPrli (i=0, 1, 2, . . . , N) in the L channel image. A luminance profilegenerated at the corresponding position in the L channel image smoothedat the large scale and differentiated is shown in 706 of FIG. 7.

The approximate feature identifying unit 1331 searches luminance valuesfrom the center line position G0 on the luminance profile in the Rchannel image or L channel image differentiated after being smoothed atthe large scale, and identifies approximate wall position candidates inthe following manner. That is, the approximate feature identifying unit1331 identifies approximate wall position candidates by acquiring amaximum value Gmax in the R channel image and a minimum value Gmin inthe L channel image. A luminance profile Gprli1 in the L channel imagesmoothed at the large scale and differentiated is illustrated in 706 ofFIG. 7. The luminance profile Gprli1 is referenced to identifyapproximate position candidates for the left wall.

(Step S541)

The membrane feature identifying unit 1333 identifies membraneboundaries in the following procedure.

(i) At each position on the vessel center line in the imagedifferentiated after being smoothed at the small scale in (i) of stepS531, a luminance profile is acquired on a line segment substantiallyperpendicular to the vessel center line. Also, the approximate wallpositions identified in (iv) of step S531 are acquired.

(ii) On the luminance profile acquired in (i), three local maximumpoints near the approximate wall position on the right side and threelocal minimum points near the approximate wall position on the left sideare detected, and the detected points are defined as membrane boundarycandidate points. Then, after grouping the membrane boundary candidatepoints into six groups, the first-order moment (i.e., “distance from thecenter line”×“luminance value”) is calculated for each membrane boundarycandidate point. Here, a luminance value in the R channel image isreference as the luminance value of the membrane boundary candidate forthe right wall, and a luminance value in the L channel image isreference as the luminance value of the membrane boundary candidate forthe left wall. A predetermined percentage of the moment values at thetop and bottom in each group are considered as outliers, and themembrane boundary candidate points having the outliers are removed.Then, the remaining membrane boundary candidate points are interpolatedin the wall running direction to identify the membrane boundary. Anyknown technique may be used to determine the outliers.

Details of the operation performed in step S541 of FIG. 5B will bedescribed with reference to FIG. 7 and the flowchart of FIG. 13B. Notethat step S1341 will not be described here, as it is the same as thecorresponding step in the first embodiment.

(Step S1311)

At each position on the vessel center line in the image differentiatedafter being smoothed at the small scale, a luminance profile along aline segment perpendicular to the vessel center line and an approximatewall position are acquired. A luminance profile Gprli2 (i=0, 1, 2, . . ., N) in the L channel image smoothed at the small scale anddifferentiated is illustrated in 707 of FIG. 7.

(Step S1321)

In the luminance profile acquired in step S1311, the membrane featureidentifying unit 1333 select three extremal points near the approximatewall position to identify membrane boundary candidate positions. Forexample, in the case of the L channel image, in the luminance profileGprli2 shown in 707 of FIG. 7, the membrane feature identifying unit1333 selects three local minimum values (Glmin_in, Glmin_out, and asmall local minimum value between them) near the approximate wallposition Gmin to identify membrane boundary candidate positions. Forexample, in the case of an artery having a vessel diameter of about 100μm, a region from Glmin_in to the small local minimum point correspondsto an intima, and a region from the small local minimum point toGlmin_out corresponds to a media. In this case, Glmin_in is an innerboundary candidate for the intima of the left wall, the small localminimum point is a boundary candidate for the boundary between theintima and a membrane having wall cells of the left wall, and Glmin_outis an outer boundary candidate for the membrane having wall cells of theleft wall.

(Step S1331)

The image processor 130 groups the membrane boundary candidatesidentified on the luminance profiles in step S1321 into the followingsix groups:

(i) outer boundary candidate group for the membrane having wall cells ofthe left wall,

(ii) boundary candidate group for the boundary between the intima andthe membrane having wall cells of the left wall,

(iii) inner boundary candidate group for the intima of the left wall,

(iv) outer boundary candidate group for the membrane having wall cellsof the right wall,

(v) boundary candidate group for the boundary between the intima and themembrane having wall cells of the right wall, and

(vi) inner boundary candidate group for the intima of the right wall.

Then, the image processor 130 determines outliers in each boundarycandidate group and removes them. In the present embodiment, thefirst-order moment (i.e., “luminance value”×“distance from the vesselcenter line”) is calculated for each boundary candidate group. Then, atop Tt2% and a bottom Tb2% are considered as outliers, and membraneboundary candidates having the corresponding moment values are removed.Here, a luminance value in the non-confocal image (i.e., R channel imagefor membrane boundary candidates for the right wall, and L channel imagefor membrane boundary candidates for the left wall) is referenced as theluminance value.

(Step S551)

On the basis of the positions of the membrane boundaries forming eachvascular wall identified in step S541, the measuring unit 135 measuresthe distribution of wall thickness or membrane thickness along therunning of the vessel, and the distribution of index value related tothe wall thickness or membrane thickness.

In the present embodiment, the measuring unit 135 calculates the wallthickness of the detected wall, the inner and outer diameters of thevessel, the index value (wall-to-lumen ratio) related to the wallthickness, the intima thickness, and the media thickness, and thendetermines the average value, standard deviation, and maximum andminimum values. These statistical values may be calculated not only forthe entire image, but also for each branch vessel. The wall thicknessand the membrane thickness may be calculated for each side (i.e., rightor left side in the vessel running direction) within the branch vessel,or may be calculated for each small region.

(Step S561)

The display control unit 136 displays the acquired images, the positionsof detected membrane boundaries, and the measurement result (wall andmembrane thicknesses, and index values related to the wall and membranethicknesses) on the monitor 305. In the present embodiment, the displaycontrol unit 136 displays the following (i) to (iv) on the monitor 305:

(i) non-confocal moving image (I1 in FIG. 9A),

-   -   image obtained by selecting frames corresponding to a specific        phase of a pulse wave and averaging the frames (12 in FIG. 9A),        and    -   image extracting the lumen of the vessel for comparison (13 in        FIG. 9A);

(ii) detected membrane boundary positions;

(iii) graph showing the wall and membrane thicknesses measured along therunning of the vascular wall or the index values related to the wall andmembrane thicknesses; and

(iv) map showing the distribution of the wall and membrane thicknessescalculated for each small region or the index values related to the walland membrane thicknesses.

For (iv), the display control unit 136 associates the calculated valueswith a color bar and displays them in color.

For smoothing and differentiating a luminance profile, the presentembodiment applies a smoothing filter and a differentiating filter to anon-confocal image to acquire a luminance profile on the smoothed anddifferentiated image. However, the present invention is not limited tothis. For example, in the procedures illustrated in FIGS. 13C and 13D, aluminance profile is acquired in a direction crossing a vessel in anon-confocal image. The present invention also includes the case ofgenerating a smoothed and differentiated luminance profile by repeatingthe process which involves linearly approximating the luminance profilein each small window and outputting the slope of the regression linewhile moving the small window at regular intervals.

In the present embodiment, two types of non-confocal images are eachsmoothed at different scales and differentiated, so as to identify theapproximate wall positions and the membrane boundaries on the basis offeature points on the luminance profiles in images obtained by smoothingthe L channel image (for the left wall) and the R channel image (for theright wall) at different scales and differentiating the resultingimages. However, the present invention is not limited to this. Forexample, the present invention also includes the case of identifying theapproximate positions of vascular walls and the membrane boundaries inthe following manner. That is, the present invention also includes thecase of identifying the approximate positions of vascular walls and themembrane boundaries by referring to luminance profiles in imagesobtained by smoothing a single non-confocal image (R+L image or imagecaptured with a large pinhole diameter), such as that shown in FIG. 6M,at different scales and differentiating the resulting images.

The present embodiment deals with the case in which, when a moving imageobtained after registration is smoothed at different scales, thesmoothing is performed with different filter sizes after frameaveraging. However, the present invention is not limited to this. Forexample, the present invention also includes the case in which an imageobtained by frame averaging alone is used as an image smoothed at asmall scale, and an image obtained by in-plane smoothing after frameaveraging is used as an image smoothed at a large scale. The presentinvention also includes the case in which two-dimensional imagessmoothed at different scales are generated by applying athree-dimensional smoothing filter having different filter sizes in thein-plane direction to a moving image obtained after inter-frameregistration. In the case of performing frame averaging, framescorresponding to a specific phase of a pulse wave may be selected andaveraged, so as to prevent the positions of membrane boundaries formingthe vascular wall from being changed by the impact of cardiac beats.

Although the present embodiment deals with the case of automaticallyacquiring a vessel center line using a morphological filter, the presentinvention is not limited to this. For example, the present inventionalso includes the case of manually setting a vessel center line byacquiring, from the instruction acquiring unit 140, the position of thevessel center line specified by the operator using the keyboard 306 orthe mouse 307.

With the configuration described above, in images obtained by smoothinga retinal vascular wall image at different scales and differentiatingthe resulting images, the retinal vascular wall image being capturedusing an SLO apparatus configured to acquire non-confocal images, theimage processing apparatus 10 identifies membrane boundaries on thebasis of feature points on luminance profiles acquired in a directioncrossing the retinal vessel.

Thus, a membrane region forming a vascular wall can be easily androbustly identified in an image of the subject's eye.

Other Embodiments

Although the image acquiring section 110 includes both the confocal dataacquiring unit 111 and the non-confocal data acquiring unit 112 in theembodiments described above, the image acquiring section 110 does notnecessarily need to include the confocal data acquiring unit 111 as longas it is configured to acquire two or more types of non-confocal data.

Although the embodiments described above deal with the cases whereapproximate wall positions and wall boundaries are identified on thebasis of feature points on smoothed and differentiated luminanceprofiles, the approximate wall positions and the wall boundaries may beidentified on the basis of feature points on luminance profiles inimages smoothed at different scales. A luminance profile Pri beforesmoothing is shown in 701 of FIG. 7. In 701 of FIG. 7, P0 corresponds tothe vessel center, Pwl corresponds to the vascular wall on the leftside, and Pwr corresponds to the vascular wall on the right side. Forexample, to identify an approximate wall position, luminance values aresearched on the left side (or right side) of the local maximum point inthe center of the luminance profile, and a position having apredetermined ratio with respect to the difference between the maximumand minimum values (or difference between the luminance value at thevessel center P0 and the minimum value) may be identified as theapproximate wall position.

If a set scale value for smoothing is not appropriate for the image tobe processed, the number of feature points on a luminance profile in thesmoothed and differentiated image may be insufficient. Therefore, foridentifying wall boundaries, if the number of points having extremalvalues in a luminance profile on an image smoothed at a small scale anddifferentiated is less than four, the scale for smoothing may be changeduntil the number of points having extremal values reaches four. In thecase of identifying membrane boundaries, if the number of points havingextremal values in a luminance profile on a smoothed and differentiatedimage is less than six, the scale for smoothing may be changed until thenumber of points having extremal values reaches six.

To facilitate identification of a vessel region (including a vesselcenter line and wall boundaries) to be measured, the image processingapparatus 10 may include a region-of-interest setting unit configured toset at least one region of interest within a predetermined range in anacquired fundus image. The field of view of the region of interest isset to a size of 0.5 mm by 0.5 mm or less so as to include only onevessel. The length of a line segment on which the luminance distributionacquiring unit 131 generates a luminance profile may be set to besubstantially double the diameter of the vessel to be measured.

Embodiments of the present invention can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiment(s) of the present invention, andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or more ofa central processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. An image processing apparatus comprising: aprocessing unit configured to perform, on a luminance distributionacquired in at least one direction crossing a vessel region in an imageof an object, a first smoothing operation with a first size and a secondsmoothing operation with a second size smaller than the first size; afirst identifying unit configured to identify a position of a vascularwall in the image on the basis of the luminance distribution obtained byperforming the first smoothing operation; and a second identifying unitconfigured to identify positions of inner and outer boundaries of thevascular wall in the image on the basis of the luminance distributionobtained by performing the second smoothing operation.
 2. The imageprocessing apparatus according to claim 1, wherein the processing unitperforms a first differentiating operation which is differentiation on aluminance distribution obtained by performing a smoothing operation witha first filter size corresponding to the first size, and performs asecond differentiating operation which is differentiation on a luminancedistribution obtained by performing a smoothing operation with a secondfilter size corresponding to the second size; the first identifying unitidentifies, as the position of the vascular wall, a feature point in theluminance distribution obtained by performing the first differentiatingoperation; and the second identifying unit identifies, as the positionsof the inner and outer boundaries of the vascular wall, a plurality offeature points in the luminance distribution obtained by performing thesecond differentiating operation.
 3. The image processing apparatusaccording to claim 1, wherein the processing unit performs, on theluminance distribution acquired in the direction crossing the vesselregion in the image, a first linear approximation with a first windowsize corresponding to the first size and a second linear approximationwith a second window size corresponding to the second size; the firstidentifying unit identifies, as the position of the vascular wall, afeature point in the luminance distribution obtained by performing thefirst linear approximation; and the second identifying unit identifies,as the positions of the inner and outer boundaries of the vascular wall,a plurality of feature points in the luminance distribution obtained byperforming the second linear approximation.
 4. The image processingapparatus according to claim 1, further comprising an image acquiringunit configured to acquire, as the image, an image obtained byperforming a subtraction or division operation between a plurality oftypes of non-confocal images obtained by receiving a plurality of lightbeams produced by splitting scattered light from the object.
 5. An imageprocessing apparatus comprising: a processing unit configured toperform, on a luminance distribution acquired in at least one directioncrossing a vessel region in an image of an object, a smoothing operationwith a first size and a second size smaller than the first size; a firstidentifying unit configured to identify, as positions of a firstvascular wall and a second vascular wall, a plurality of feature pointsin the luminance distribution obtained by performing the smoothingoperation with the first size; and a second identifying unit configuredto identify, as positions of inner and outer boundaries of the firstvascular wall and positions of inner and outer boundaries of the secondvascular wall, a plurality of feature points near the identifiedpositions of the first and second vascular walls in the luminancedistribution obtained by performing the smoothing operation with thesecond size.
 6. The image processing apparatus according to claim 5,wherein the second identifying unit identifies, as the positions of theinner and outer boundaries of the first vascular wall and the positionsof the inner and outer boundaries of the second vascular wall, aplurality of extremal points near the positions of the first and secondvascular walls in the luminance distribution obtained by performing thesmoothing operation with the second size.
 7. The image processingapparatus according to claim 5, wherein the second identifying unitidentifies a plurality of feature points in the luminance distributionobtained by performing the smoothing operation with the second size, asa position of an inner boundary of an intima, a position of a boundarybetween the intima and a membrane having wall cells, and a position ofan outer boundary of the membrane having wall cells for each of thefirst and second vascular walls.
 8. The image processing apparatusaccording to claim 7, further comprising a measuring unit configured tomeasure at least one of a vessel membrane thickness, a vascular wallthickness, a vessel inner diameter, and a vessel outer diameter on thebasis of at least one of the inner boundary of the intima, the boundarybetween the intima and the membrane having wall cells, and the outerboundary of the membrane having wall cells.
 9. The image processingapparatus according to claim 4, further comprising a measuring unitconfigured to measure at least one of a vascular wall thickness, avessel inner diameter, and a vessel outer diameter using the positionsof the outer and inner boundaries of the first vascular wall and thepositions of the outer and inner boundaries of the second vascular wall.10. The image processing apparatus according to claim 5, furthercomprising a region-of-interest setting unit configured to set at leastone region of interest with a size of 0.5 mm by 0.5 mm or less in theimage to identify the vessel region in the image.
 11. The imageprocessing apparatus according to claim 5, further comprising an imageacquiring unit configured to acquire, as the image, an R channel imageand an L channel image which are a plurality of types of non-confocalimages obtained by receiving a plurality of light beams produced bysplitting scattered light from the object, wherein the secondidentifying unit identifies the positions of the inner and outerboundaries of the first vascular wall using one of the R channel imageand the L channel image, and identifies the positions of the inner andouter boundaries of the second vascular wall using the other of the Rchannel image and the L channel image.
 12. An image processing apparatuscomprising: a first identifying unit configured to identify a positionof a vascular wall in an image of an object on the basis of a luminancedistribution acquired in at least one direction crossing a vessel regionin the image; and a second identifying unit configured to identify aplurality of feature points near the identified position of the vascularwall in the luminance distribution as positions of inner and outerboundaries of the vascular wall in the image.
 13. An image processingmethod comprising: a processing step of performing, on a luminancedistribution acquired in at least one direction crossing a vessel regionin an image of an object, a first smoothing operation with a first sizeand a second smoothing operation with a second size smaller than thefirst size; a first identifying step of identifying a position of avascular wall in the image on the basis of the luminance distributionobtained by performing the first smoothing operation; and a secondidentifying step of identifying positions of inner and outer boundariesof the vascular wall in the image on the basis of the luminancedistribution obtained by performing the second smoothing operation. 14.A non-transitory computer-readable storage medium storing a programcausing a computer to execute each step of the image forming methodaccording to claim
 13. 15. An image processing method comprising: aprocessing step of performing, on a luminance distribution acquired inat least one direction crossing a vessel region in an image of anobject, a smoothing operation with a first size and a second sizesmaller than the first size; a first identifying step of identifying, aspositions of a first vascular wall and a second vascular wall, aplurality of feature points in the luminance distribution obtained byperforming the smoothing operation with the first size; and a secondidentifying step of identifying, as positions of inner and outerboundaries of the first vascular wall and positions of inner and outerboundaries of the second vascular wall, a plurality of feature pointsnear the identified positions of the first and second vascular walls inthe luminance distribution obtained by performing the smoothingoperation with the second size.
 16. A non-transitory computer-readablestorage medium storing a program causing a computer to execute each stepof the image forming method according to claim
 15. 17. An imageprocessing method comprising: a first identifying step of identifying aposition of a vascular wall in an image of an object on the basis of aluminance distribution acquired in at least one direction crossing avessel region in the image; and a second identifying step of identifyinga plurality of feature points near the identified position of thevascular wall in the luminance distribution as positions of inner andouter boundaries of the vascular wall in the image.
 18. A non-transitorycomputer-readable storage medium storing a program causing a computer toexecute each step of the image forming method according to claim 17.